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This  paper  describes  how  to  do  useful,  nontrivial 
image  processing  tasks  interactively  using  only  a  standard 
alphanumeric  CRT  terminal,  or.  even  a  teletype.  Only  an 
ordinary  time-sharing  system  is  required;  there  is  no  need 
for  a  dedicated  computer  or  channel,  or  even  for  special 
priority  on  the  system. 


The  support  of  the  Air  Force  Office  of  Scientific  Research 
under  Contract  F44620-72-C-0062  is  gratefully  acknowledged, 
as  is  the  help  of  Eleanor  B.  Waters  in  preparing  this  paper. 


Introduction 


The  advantages  of  interactive  processing  over  batch  processing 
are  by  now  well  established.  When  working  in  an  interactive  mode, 

the  programmer  obtains  quick  responses  to  his  actions,  and  does 
not  have  to  reconstruct  his  line  of  reasoning  each  time  a  response 
is  obtained.  This  is  especially  important  when  the  data  being  processed 
are  graphical  or  pictorial;  it  is  of  great  benefit  to  be  able  to  see 
the  results  of  each  processing  step  displayed  immediately.  In 
addition,  it  is  crucial  to  be  able  to  point  to  objects  in  th<  dis¬ 
play,  or  outline  regions  in  the  display,  in  such  a  way  that  the 
computer  knows  which  objects  or  regions  are  intended;  this  ability 
is  impossibly  cumbersome  to  achieve  in  batch  mode. 

Low-cost  interactive  graphics  terminals  are  now  widely  avail¬ 
able,  but  one  hears  much  less  about  interactive  image  processing 
systems.  Of  the  available  computer  image  displays,  only  the  more 
expensive  permit  any  sort  of  interaction,  such  as  pointing,  out¬ 
lining,  or  selective  modification. 

This  paper  describes  an  approach  to  interactive  image  processing 
using  only  a  standard  alphanumeric  CRT  terminal — or  if  necessary, 
even  a  teletype.  This  approach  can  be  implemented  on  any  ordinary 
time-sharing  system;  it  does  not  require  a  dedicated  computer,  a 
dedicated  channel,  or  even  special  priority  on  the  system. 
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Irout 

Digitized  images,  even  of  moderate  size,  contain  enormous 
numbers  of  bits;  for  example,  a  commercial  TV  picture  contains 
about  500  by  500  resolvable  points,  and  if  we  represent  the  gray 
level  of  each  point  by  a  6-bit  number,  we  have  lh  million  bits  in 
the  picture.  For  this  reason,  it  has  often  been  suggested  that, 
when  doing  image  processing  on  a  computer,  one  should  not  digitize 
the  entire  image  and  input  it  to  the  computer;  rather,  one 
should  allow  the  computer  to  control  a  scanning  device  which  can 
read  and  digitize  selected  portions  of  the  image  on  demand.  If 
this  suggestion  is  accepted,  it  implies  a  major  hardware  expense 
before  the  image  processing  itself  can  even  begin. 

Fortunately,  there  are  alternatives  to  the  computer-controlled 
scanner  approach,  provided  that  one  is  willing  to  trade  I/O  time 
for  hardware  cost.  A  digitized  TV  picture  occupies  only  a  few  yards 
of  magnetic  tape;  one  can  store  many  such  pictures  on  a  single 
tape.  If  desired,  the  picture  can  be  transferred  to  disk  or  drum 
storage  before  beginning  its  actual  processing.  I/O  limitations 
may  make  it  somewhat  slower  to  access  pictures  from  disk  than 
to  access  them  by  controlling  a  scanner,  but  this  is  a  modest 
price  to  pay  in  place  of  having  to  buy  or  build  the  scanner. 

There  are  also  counterarguments  to  the  scanner  concept  itself. 
Information  obtained  by  the  scanner  may  not  be  perfectly  repeatable. 
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since  analog  signals  are  involved;  in  some  circumstances,  this 
may  be  highly  objectionable.  In  many  cases,  too,  the  processing 
to  be  done  on  the  input  image  requires  many  times  more  storage 
than  does  that  image  itself,  so  that  the  storage  saving  resu1  .ng 
from  being  able  to  use  the  image  as  a  memory  is  negligible.* 

Digitized  images  on  magnetic  tape  are  widely  available 
from  many  sources,  and  digitizing  services  are  also  available,  at 
a  cost  of  only  a  few  dollars  per  image.  Thus  anyone  wishing  to 
undertake  experimental  image  processing  can  provide  himself  with 
a  data  base  very  cheaply.  Experience  suggests,  moreover,  that 
most  experimenters  will  need  only  a  few  digital  images  for  pro¬ 
gram  development  and  feasibility  testing.  In  the  early  stages  of 
image  processing  research,  one  usually  wants  to  try  out  many 
techniques  on  the  same  image,  not  one  technique  on  many  images. 

One  can  even  do  non -trivial  image  processing  research  on 
computer-created  images  (e.g.,  defined  by  stochastic  processes). 

In  fact,  it  is  wise  to  develop  and  test  one's  image  processing 
software  using  simple  test-pattern  images,  so  that  there  is  no 
doubt  what  the  results  of  the  operations  are  supposed  to  look  like. 

♦The  processing  is  inexpensive  in  terms  of  storage  only  when  it 
involves  relatively  rare  events  that  can  be  detected  in  the  image 
as  it  is  scanned,  so  that  most  of  the  image  need  not  be  stored. 


Whon  processing  images  interactively,  one  usually  need 
not  look  at  the  entire  image  after  each  step.  Display  of 
small  pieces  should  generally  be  adequate  for  checking  that 
the  correct  image  has  been  read  in,  or  that  a  step  has  been 
executed  correctly.  An  example  is  shown  in  Figure  1;  it  is  a 
72-by-44-point  image,  where  each  point  has  one  of  32  shades 
of  gray. 

An  output  such  as  that  in  Figure  1  requires  grayscale  dis¬ 
play  hardware  that  is  not  normally  available  to  an  ordinary 
time-sharing  system  user.  Moreover,  the  time  required  to  out¬ 
put  Figure  1,  say  on  a  300-baud  line,  would  be  about  two  min¬ 
utes  (assuming  one  character  per  image  point) .  Given  a  display 
without  grayscale  capability,  if  the  user  attempted  to  achieve 
grayscale  by  exciting  display  points  repeatedly,  the  time  would 
become  even  greater.  Moreover,  the  resulting  display  is  small 
(Figure  1  already  involves  considerable  defocus) ,  making  it  hard 
to  interact  with  the  displayed  image  accurately. 

Most  of  these  difficulties  can  be  avoided  by  using 
arrays  of  alphanumeric  characters  for  image  output.  One  now 
needs  only  an  ordinary  alphanumeric  terminal  (even  a  teletype 
will  do) ,  though  of  course  a  graphics  terminal  too  can  be  used 
in  this  way.  Gray  shades  are  represented  using  characters  that 
have  various  ratios  of  character  area  to  background  area,  ranging 
from  blank  to  (say)  W.  One  can  construct  an  adequate  8-step 
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y  scale  in  this  way  with  single  characters;  and  if  overstrike 
is  permitted,  a  32-step  scale  can  be  obtained.  Figure  2  shows 
the  image  of  Figure  1  output  in  this  way,  and  the  character  sets 
used.  The  overstriking  is  achieved  by  eliminating  the  line  feed 
sent  by  the  computer.  Unfortunately,  one  cannot  do  "overstriking" 
on  a  CRT  terminal,  but  it  can  be  done  on  the  CRT’s  auxiliary 
hard-copy  printer.  There  is  a  vertical/hori2ontal  scale  distortion 
of  about  3:2,  but  this  is  not  objectionable  for  most  purposes. 

(One  can,  of  course,  also  use  non-gray  scale  alphanumeric  output 
(e.g.,  gray  levels  0, ..,31  =  blank,  1,...,9,  A,...,V/if  one  wishes  to 
read  gray  levels  of  individual  points  rather  than  see  the  points 
as  an  image;  see  Figure  2e.) 

The  overstruck  gray  scale  is  much  better  than  the  single¬ 
character  scale,  but  it  has  the  disadvantage  of  being  far  more 
time  consuming.  Outputting  a  72-by-40  single-character  image  on 
a  CRT  terminal  operating  at  1200  baud  takes  less  than  30  seconds;  but 
outputting  an  overstruck  image  of  the  same  size  on  a  teletype  at 
110  baud  takes  nearly  ten  minutes,  which  is  impractically  tedious. 

It  the  single-character  scheme  is  used,  it  is  important  to 
pick  the  gray  level  ranges  that  correspond  to  the  characters 
carefully.  A  good  rule  of  thumb  is  to  pick  the  ranges  to  contain 
numbers  of  picture  points  that  are  as  equal  as  possible. 

The  8-level  grayscale  of  Figure  2a-b  was  designed  in 


this  way.  Figure  3a  shows  the  gray-level  histogram  for  the  image 
in  Figure  1,  as  well  as  the  ranges  used  to  produce  the  8-level 
version  in  Figure  2.  Gray  level  range  and  threshold  selection 
can,  of  course,  be  done  interactively.  The  effects  of  a  poor 
choice  of  ranges  is  shown  in  Figure  3b.* 

Display  of  small  pieces  of  an  image  is  usually  adequate  for 
checking  results,  and  even  for  interacting  by  pointing  or  out¬ 
lining,  since  the  objects  or  regions  to  be  outlined  will  usually 
be  small  relative  to  the  entire  image.  At  times,  as  in  the 
examples  in  this  paper,  output  of  small  pieces  is  all  that  is 
needed.  If  output  of  entire  images  is  required,  overstrike  on  the 
line  printer  can  be  used  (remembering  to  override  the  page-skip!); 
a  500-by-500  point  image  can  be  output  in  four  vertical  strips, 
each  a  page  wide  and  about  six  pages  long.  This  requires  much 
tearing  and  pasting,  but  has  the  merit  of  requiring  no  special 
equipment.  If  a  microfilm  output  device  with  nalf-tcne  capability 
is  available,  it  provides  a  far  more  compact  output  format. 
Alternatively,  one  can  output  images  on  tape,  and  take  advantage 
of  commercially  available  tape -to- image  equipment. 

♦Figures  2a  and  3b  are  best  compared  by  viewing  from  a  distance. 
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Interaction 

The  ability  to  execute  an  image  processing  operation 
and  display  the  results  already  constitutes  a  high  degree  of  inter¬ 
active  capability,  since  the  user  can  determine  what  to  do  next 
in  near-real  time.  The  chief  facility  still  lacking  is  the  ability 
to  point  to  objects  in  the  image.  It  should  be  realized  that  in 
image  processing,  pointing  can  usually  designate  only  a  single 
image  point,  since  the  image  has  no  underlying  structural  des¬ 
cription  that  is  known  to  the  computer,  as  it  has  in  computer  graphics. 

J 

In  order  to  designate  objects  or  regions  in  an  image,  one  must  out¬ 
line  these  rather  than  simply  point  to  them. 

Region  outlining  can  be  done  straightforwardly  on  either  an 
alphanumeric  CRT  display  or  teletype  printout  of  an  image.  In  the 
CRT  case,*  the  piece  of  picture  in  question  is  displayed  by  the 
computer  in  non-protected  mode.  The  user  switches  the  terminal 
to  "batch  mode"  and  uses  the  terminal  cursor  controls  to  position 
the  cursor  on  the  top  ro w  of  the  region  to  be  outlined.  At  each 
point  where  the  region  boundary  intersects  the  row,  the  user  types 
a  character  (different  from  the  one  already  typed).  He  then  moves 
the  cursor  to  the  end  of  the  row  and  transmits  the  "revised"  line. 

The  computer  can  now  compare  this  to  the  original  line  and  store 
the  locations  of  the  differences.  When  all  rows  containing  region 

*The  exact  procedure  would  vary  somewhat,  depending  on  the 
particular  terminal  used. 


boundary  points  have  been  processed  in  this  way,  the  computer  can 
construct  the  complete  region  outline  (and  display  it,  if  desired) . 

The  procedure  on  a  teletype  is  similar,  but  more  tedious: 
the  user  informs  the  computer  that  he  is  about  to  input  an  image 
the  same  size  as  the  displayed  one.  He  then  rolls  the  teletype 
paper  back  to  the  first  line  of  the  displayed  image,  and  enters 
carriage  returns  until  the  top  row  of  the  region  is  reached.  For 
rows  that  hit  the  region,  the  user  spaces  over  to  the  region  bound¬ 
ary  points  and  overstrikes  these  points  with  an  arbitrary  character. 
Rolling  the  paper  back  simply  establishes  visual  registration  of 
the  border  and  picture  for  the  user;  the  computer  is  unaware  of 
it,  but  can  still  register  the  two  because  they  are  the  same  size. 

Both  the  CRT  and  teletype  versions  of  the  outlining  procedure 
can  be  designed  to  permit  correction  of  errors  by  re-inputting  only 
the  corrected  border  points,  without  having  to  re-outline  the  entire 
region.  In  any  case,  they  are  slow  enough  that  errors  are  unlikely. 
The  alphanumeric  display  functions  as  a  half-tone  which 

is  coarse  enough  to  permit  precise  outlining,  yet  provides  suffic¬ 
ient  grayscale  to  permit  viewing  the  output  as  an  image  rather  than  as 
an  array  of  discrete  dots.  The  outlining  procedure,  it  will  be 
noted,  requires  no  special  inte erupt  priority;  outlines  are  trans¬ 
mitted  row  by  row  exactly  as  ir  t.ie  ordinary  use  of  the  terminal. 
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A  example  of  the  use  of  outlining  is  given  in  Figure 
4,  which  shows  an  outline  in  register  with  the  picture  of 
Figures  2-3.  The  display  of  the  outlined  region  is  done 
by  a  program  which  determines  all  points  of  the  image  that 
lie  on  or  inside  the  outline.  Once  the  outlined  region 
has  been  extracted  from  its  background,  one  can  compute 
properties  of  the  region,  relating  to  its  shape,  texture, 


etc . 
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Software 

The  nature  of  the  software  used  in  an  interactive 
image  processing  system  will  depend  greatly  on  the  ma¬ 
chine  being  used,  the  programming  talent  available,  and 
the  types  of  processing  operations  to  be  performed. 

Several  major  packages  of  image  processing  programs 
are  available.  Two  notable  examples  are  Vicar,  developed 
by  California  Institute  of  Technology's  Jet  Propulsion 
Laboratory,  and  PAX,  developed  by  the  Universities  of  Il¬ 
linois  and  Maryland.  These  packages  are  usually  embedded 
in  a  high-level  language  such  as  Fortran.  The  examples 
given  in  this  paper  were  programmed  in  PAX. 
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Historical  Notes 

Most  of  the  major  image  processing  research  groups 
have  developed  interactive  systems,  but  very  few  of  these 
have  been  documented  in  the  published  literature.  A 
highly  nonrepresentative  set  of  examples  is  [1-3],  On 
the  use  of  overstruck  characters  to  represent  grayscale 
see  [4]. 

PAX  is  a  collection  of  over  100  basic  image  processing 
routines  that  can  be  called  from  Fortran  programs  [5]. 
Originally  a  simulator  for  the  ILLIAC  III  computer,  ver¬ 
sions  of  PAX  for  several  different  machines  are  available, 
notably  the  IBM  7094  and  360/370  (50  and  above) ;  Uni vac 
1108;  CDC  3600  and  6fo00;  and  DEC-10. 
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Figure  2.  Use  of  alphanumeric  arrays  to  represent 
images . 

a)  Image  of  Figure  1,  with  gray  levels 
represented  by  characters  as  shown 
in  (b) . 


Gray  level (s) 


Character 


0-3 

4 

5 

6 

7-8 

9-11 

12-16 

17-31 


blank 

+ 

I 

A 

* 

W 


Figure  2b.  Characters  used  to  represent 
gray  levels  in  (a) . 
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.  Same  image,  with  gray  levels  represented  by 
pairs  of  overstruclc  characters  as  shown 
in  (a) . 
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Figure  2e,  Same  image,  with  gray  Levels  0,...,bl 
represented  by  blank,  9,  A,..., 
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Figure  3a.  Gray  level  histogram  for  Figure  1;  only  levels 
0,  ...,23  occur.  Table  shov/c  ranges  used  in 
Figure  2a . 
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Figure  3b.  Effect  of  a  poor  choice  of  gray  level 
ranges;  0-3  -  blank,  4-7  ~  8-11  = 

12-15  =  +,  16-19  «  I,  20-23  =  A, 

24-27  -  *,  28-31  -  W.  Note  that  *  and  W 
are  never  used. 
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Figure  4a.  Outline  overstrucK  on  a  printout  of  Figure  2e 
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Figure  4b.  Printout  of  the  outline  alone, 
as  a  check. 
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Figure  4c.  Printout  of  the  outlined  region 
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